The present invention relates to digital systems, and, more particularly to an apparatus and method for adaptively generating stimulus within a digital system during a design verification operation.
Verifying the functional correctness of complex digital systems poses a major challenge for today""s verification engineers. The extremely large number of possible states present in today""s complex digital systems and the fact that some of the possible states are difficult to exercise make it a practical impossibility to completely exercise all potential states. Design verification is an example of an xe2x80x9cNP-complete problemxe2x80x9d (i.e., the amount of resources needed to solve the problem grows as an exponent of the problem size). Two major approaches have been used to by verification engineers to verify the behavior of digital systems: 1) formal verification and 2) simulation based verification.
FIG. 1 is a simplified block diagram of a formal verification process, indicated generally at 20. Formal verification process 20 employs formal methods (e.g., mathematical models) to verify the functional correctness of a digital system 22. A verification engineer first creates a mathematical model 24 representing digital system 22, then performs a series of mathematical operations 26 on mathematical model 24 to verify digital system 22, generating a set of verification results 26. In contrast to a simulation based verification approach, formal verification process 20 provides a complete check of the design, rather than checking only data sets selected by the test engineer. Formal verification process 20 highlights subtle errors that occur under unusual circumstances, and it eliminates the time necessary to create test data (e.g., test vectors). In formal verification, there are two general types of checks that can be performed: equivalence checking and property checking.
Equivalence checking is a collection of techniques used to demonstrate that two designs are equivalent. One definition of xe2x80x9cequivalentxe2x80x9d states that all registers within the two designs must have the identical content, and the outputs must be identical at the end of each clock cycle. This rigid definition is limiting in practice, as the contents of the registers change slightly as the design proceeds. In an alternative less rigid definition of xe2x80x9cequivalentxe2x80x9d, the contents of registers in a first design are a simple function of the contents of registers in a second design at the end of each clock cycle. In a yet another loose definition of xe2x80x9cequivalentxe2x80x9d, the outputs of a first design and a second design must merely go through the same transitions over time.
Property checking encompasses a collection of techniques that allow a verification engineer to resolve design issues, such as: protocols for buses, caches, and communications links. Property checking is generally applied prior to design synthesis to check that a register transfer level (RTL) design behaves in an expected way.
Formal verification techniques are still in their infancy and are severely limited by the size of a subsystem that can be effectively verified. Thus, a simulation based verification approach is more commonly employed to verify the behavior of a complex chip (e.g., microprocessor) in a system environment having multiple interacting components.
FIG. 2 is a simplified block diagram of a simulation based verification process, indicated generally at 40. Simulation based verification process 40, verifies a digital system 42 by generating various sequences of states (also referred to as events) with a test generator 44. Examples of events include: read/write operations to various memory addresses, queues, and buffers within digital system 42. Test generator 44 generates events by creating sequences of test vectors, applying the sequences of test vectors to digital system 42 or other device under test (DUT), capturing a set of verification results 46 generated by the application of the sequences of test vectors, and determining whether the responses to the applied test vectors (i.e., verification results 46) are correct. A simulation based verification, such as simulation based verification process 40, typically employs an exhaustive 48, random 50, and/or directed random 52 testing methodology.
Generating an exhaustive set of test vectors 48 is often impractical to cover the complexity found in modem digital systems. The generation of a set of random test vectors 50 often creates a xe2x80x9cscattergunxe2x80x9d effect, where the most critical areas of the design receive no more testing than less critical areas of the design.
As a result, some test generators, such as test generator 44, create a set of directed random test vectors controlled by a set of user-defined verification directives 54. The set of verification directives xe2x80x9cdirectsxe2x80x9d the generation of test vectors to test areas of particular interest in digital system 42. One example, user-defined verification directive within a configuration file provides a specific range of addresses (i.e., address distribution) within the system where the test vectors are to be directed, as indicated at 56. Another example user-defined verification directive specifies that instructions of a specific type be used during the verification, as indicated at 58. Another example user-defined verification directive specifies the ratio of read and write operations to be performed during testing, as indicated at 60. These user-defined verification directives 54 serve as fine tuning xe2x80x9cknobsxe2x80x9d for the test generation process.
In such an environment, the test generator operates in an essentially nonadaptive mode where user-defined verification directives 54 are assigned a priori and no feedback is obtained from digital system 42 to determine if an event was hit or is likely to be hit. In other words, the fine tuning xe2x80x9cknobsxe2x80x9d (i.e., user-defined verification directives) are locked into position before the verification process begins, and remain unalterable throughout the verification process. Thus, the non-adaptive nature of the test generation process results in unnecessarily long test sequences, manual intervention, and low event coverage.
In view of the above, there is a need for a function to adaptively generate test vectors based on feedback of internal state information represented within digital system during verification. The function should utilize a set of user-defined verification directives for directing the generation of the test vectors to areas of interest within the digital system in view of the internal state information received from the digital system.
The present invention provides an apparatus for adaptively generated test vectors used to verify the behavior of a digital system. The functional testing system includes a hardware model representing a relatively low-level software characterization of the digital system. The functional testing system also includes a hardware emulator representing a relatively high-level software characterization of the digital system. The hardware emulator is capable of dynamically representing internal model state information. The functional testing system further includes a test generator for generating and applying a plurality of input vectors to the hardware model and the hardware emulator to produce a directed behavior. Finally, the functional testing system includes one or more verification directives which characterize attributes desired in the functional testing system and which direct the behavior of the test generator. The test generator accesses the internal model state information represented within the hardware emulator to adaptively generate the plurality of input vectors as directed by the one or more verification directives.
In one embodiment, the adaptively generated plurality of input vectors may generate one or more transactions within the hardware model. In some instances, the generated transactions are related, and form an ordered sequence of transactions. The adaptively generated input vectors may produce transactions which conflict with a pending internal state of the digital system.
In one embodiment, the functional testing system of the present invention includes queue management directives, address conflict directives, and system behavior directives. In one embodiment of the present invention, the test generator accesses the internal model state information represented within the emulator via a software procedural interface.
In one embodiment, the present invention also provides a method for adaptively generate test vectors to verify the behavior of a digital system. The method begins by defining a hardware model representing a relatively low-level software characterization of the digital system. Next, a hardware emulator is defined which represents a relatively high-level software characterization of the digital system, wherein the hardware emulator is capable of dynamically representing internal model state information. A plurality of input vectors is then generated to produce a directed behavior within the hardware model. The generation utilizes the internal model state information represented within the hardware emulator. Finally, the plurality of input vectors is applied to the hardware emulator and hardware model, producing the directed behavior.
In one embodiment, the adaptively generated plurality of input vectors generates one or more transactions within the hardware model. In some instances, the generated transactions are related, and form an ordered sequence of transactions. In some instances, the adaptively generated input vectors produce transactions which conflict with a pending internal state of the digital system. The functional testing system of the present invention includes queue management directives, address conflict directives, and system behavior directives.
In one embodiment, the present invention further provides a computer readable medium containing instructions for controlling a computer system to perform a method for adaptively generating test vectors for verifying the behavior of a digital system. The method begins by defining a hardware model representing a relatively low-level software characterization of the digital system. Next a hardware emulator is defined representing a relatively high-level software characterization of the digital system, where the hardware emulator is capable of dynamically representing internal model state information. A plurality of input vectors is then generated to produce a directed behavior within the hardware model. The generation utilizes the internal model state information represented within the hardware emulator. Finally, the plurality of input vectors are applied to the hardware emulator and the hardware model, producing the directed behavior.
The present invention offers several advantages over current functional testing systems for verifying the behavior of a digital system. The present invention provides both a relatively low-level hardware model and a relatively high-level hardware emulator for representing the digital system both structurally and behaviorally during the verification process. By dynamically monitoring feedback of internal state information from the hardware emulator during the verification process, the test generator of the present invention utilizes a set of user-defined verification directives in conjunction with the state information feedback to adaptively generate test vectors to test areas of interest within the digital system. This adaptive generation of test vectors found within the current invention requires less overhead and is easier to maintain than non-adaptive test generation systems currently in use.